OpenFabrics-suite-of-examples/blast-sr-2

The OpenFabrics suite of examples is code developed for the Programming
with OpenFabrics Software Training Course.

Copyright (c) 2011 OpenFabrics Alliance, Inc.  All rights reserved.

This software is available to you under a choice of one of two
licenses.  You may choose to be licensed under the terms of the GNU
General Public License (GPL) Version 2, available from the file
GNU_GPL_OFA.txt in the directory containing this source file, or the
OpenIB.org BSD license, available from the file BSD_for_OFA.txt in the
directory containing this source file.


This demo illustrates a simple ping-pong client server application.
It does NOT use separate threads or handlers for either cm events
or completion events.  It is essentially uses the asynchronous
verbs synchronously -- it polls for everything.

The flow for the client program is:

		1. rdma_create_event_channel()
			creates a communication channel on which to report
			asynchronous cm events
		2. rdma_create_id()
			allocates a communication identifier rdma_cm_id
			add associates it with the cm event channel
			this is passed as a parameter to subsequent
			cm calls in order to identify the connection
		3. (rdma_)getaddrinfo()
			translates human-readable DNS name or IP address and
			port number into internal addressing form
		4. rdma_resolve_addr()
			to do DNS lookup on server address and bind
			this rdma_cm_id to a local RDMA device
		6. rdma_get_cm_event()
		6. rdma_ack_cm_event()
		7. rdma_resolve_route()
			to find an RDMA route to the server
		8. rdma_get_cm_event()
		9. rdma_ack_cm_event()
		10.ibv_alloc_pd()
			allocates a protection domain (PD)
		11.ibv_create_comp_channel()
			creates a communicaton channel on which to report
			asynchronous completion events
		12.ibv_create_cq()
			creates a completion queue to hold pending events
		13.rdma_create_qp()
			to create a queue-pair (i.e., a send and a recv queue)
			and get it ready for sending and receiving
		14.ibv_reg_mr()
			to register memory areas to be used for send and recv
		15.--create valid work requests and scatter-gather elements
			to use registered memory areas in send and recv
		16.rdma_connect()
			for RDMA_PS_TCP, initiates a connection request
				to remote server
			for RDMA_PS_UDP, initiates a lookup of the
				remote QP providing the datagram service
			allows user to specify operating parameters for
			this connection:
				max outstanding RDMA read/atomic ops to/from
					remote side
				use hardware flow control
				max retries on an error
		17.rdma_get_cm_event()
		18.rdma_ack_cm_event()
	loop
		19.ibv_post_recv()
			initiate a receive to catch the reply from the server
		20.ibv_post_send()
			initiate a send to the server
		21.ibv_poll_cq()
			wait for send to complete
		22.ibv_poll_cq()
			wait for receive to comple
	endloop
		23.rdma_disconnect() -- undoes step 16
			tell the cm to break the network connection
		24.rdma_get_cm_event()
			wait for the result from the cm
		25.rdma_ack_cm_event()
		26.ibv_dereg_mr() -- undoes step 14
			unregister the memory areas used for send and recv
		27.rdma_destroy_qp() -- undoes step 13
			destroys queue pair
		28.ibv_destroy_cq() -- undoes step 12
			destroys the completion queue
		25.ibv_destroy_comp_channel() -- undoes step 11
			destroys the communication channel for completion events
		26.ibv_dealloc_pd() -- undoes step 10
			deallocates the protection domain
		27.rdma_destroy_id() -- undoes step 2
			destroys the cm_id
		28.rdma_destroy_event_channel() -- undoes step 1
			destroys the communication channel for cm events
